Rateless codes for near-perfect load balancing in distributed matrix-vector multiplication

Large-scale machine learning and data mining applications require computer systems to perform massive matrix-vector matrix-matrix multiplication operations that need be parallelized across multiple nodes. The presence of straggling nodes---computing nodes unpredictably slow down or fail---is a major bottleneck in such distributed computations. Ideal load balancing strategies dynamically allocate more tasks faster knowledge monitoring node speeds as well the ability quickly move data. Recently proposed fixed-rate erasure coding can handle unpredictable slowdown, but they ignore partial work done by nodes, thus resulting lot redundant computation. We propose rateless fountain strategy achieves best both worlds---we prove its latency is asymptotically equal ideal balancing, it performs zero Our idea create linear combinations m rows matrix assign these encoded different worker original product decoded soon slightly than row-vector products are collectively finished Evaluation on parallel computing yields much three times speedup over uncoded schemes.